The storage of datasets can become very space intensive. Some datasets can comprise large numbers of items of uniform size. The size of such a dataset increases linearly as the number of items increases. For example, a set of malicious code signatures stored by an anti-malware system typically comprises an ever increasing number of anti-malware signatures of a fixed width (e.g., 32 bits). As such a dataset becomes larger, the download and use time for the dataset is affected, and the memory footprint and storage requirements increase.
It would be desirable to address these issues beyond the solutions provided by conventional compression technologies.